Journal of Vision

The Pulfrich Effect is an illusion characterized by the misperception of the depth and 3D direction of moving objects. Interocular luminance differences cause the Classic Pulfrich effect; the darker image is processed more slowly. Interocular blur differences cause the Reverse Pulfrich effect; the blurrier image is processed more quickly. A common correction for presbyopia--monovision--intentionally induces the optical conditions that cause the Reverse Pulfrich Effect. The effect sizes, and the fact that tens of millions of people wear these corrections every day, raise concerns about public safety. However, although the impact of overall light-level (e.g., nighttime vs. daytime) on the Classic Pulfrich effect has been well-characterized, its impact on the Reverse Pulfrich effect is unknown. Here, using a custom binocular 4f tunable lens optical system that allows the decoupling of retinal illuminance and retinal blur, we report how the Classic and Reverse Pulfrich effects scale with overall light-level. Both effects increase logarithmically with decreases in light-level. These results motivate a characterization of how light level interacts with other optical factors (e.g., higher-order aberrations) that are likely to impact the Reverse Pulfrich effect, and hence the perceptual consequences of monovision corrections. Commercial disclosureV. Rodriguez Lopez: None; B. Chin: None; J. Burge: None.

We characterized the temporal dynamics of color processing using a continuous tracking paradigm by estimating temporal impulse response functions associated with tracking chromatic Gabor patches. We measured how the lag of these functions changes as a function of chromatic direction and contrast for stimuli in the LS cone contrast plane. In the same set of subjects, we also measured detection thresholds for stimuli with matched spatial, temporal, and chromatic properties. We created a model of tracking and detection performance to test if a common representation of chromatic contrast accounts for both measures. The model summarizes the effect of chromatic contrast over different chromatic directions through elliptical isoresponse contours, the shapes of which are contrast independent. The fitted elliptical isoresponse contours have essentially the same orientation in the detection and tracking tasks. For the tracking task, however, there is a striking reduction in sensitivity to signals originating in the S cones. The results are consistent with common chromatic mechanisms mediating performance on the two tasks, but with task-dependent relative weighting of signals from L and S cones.

Lightness constancy, the ability to create perceptual representations that are strongly correlated with surface albedo despite variations in lighting and context, is a challenging computational problem. Indeed, it has proven difficult to develop image-computable models of how human vision achieves a substantial degree of lightness constancy in complex scenes. Recently, convolutional neural networks (CNNs) have been developed that are proficient at estimating albedo, but little is known about how they achieve this, or whether they are good models of human vision. We examined this question by training a CNN to estimate albedo and illumination in a computer-rendered virtual world, and evaluating both the CNN and human observers in a lightness matching task. In several conditions, we eliminated cues potentially supporting lightness constancy: local contrast, shading, shadows, and all contextual cues. We found that the network achieved a high degree of lightness constancy, outperforming three classic models, and substantially outperforming human observers as well. However, we also found that eliminating cues affected the CNN and humans very differently. Humans had much worse constancy when local contrast cues were made uninformative, but were minimally affected by elimination of shading or shadows. The CNN was unaffected by local contrast, but relied on shading and shadows. These results suggest that the CNN followed an effective strategy of integrating global image cues, whereas humans used a more local strategy. In a follow-up experiment, we found that the CNN could learn to exploit noise artifacts that were correlated with illuminance in ray-traced scenes, whereas humans did not. We conclude that CNNs can learn an effective, global strategy of estimating lightness, which is closer to an optimal strategy for the ensemble of scenes we studied than the computation used by human vision.

Color supports object identification. However, two objects that differ in color under one light can appear indiscriminable under a second light. This phenomenon, known as illuminant metamerism, underlies the difficulty faced by consumers of selecting matching fabric or paint colors in a store only to find that they appear not to match under home lighting. The frequency of illuminant metamerism has been evaluated only under single, uniform illuminants. However, in real world conditions, the spectral content of light falling on an object varies with direction (Morimoto et al. 2019), meaning that a surface will sample different spectra depending on its angle within the environment. Here we used computer-graphics techniques to simulate a pair of planar surfaces placed under newly measured hyperspectral illumination maps that quantify the directional variability of real-world lighting environments. We counted the instances of illuminant metamerism that can be solved simply by viewing surfaces tilted to a different direction. Results show that most instances of illuminant metamerism can in theory be resolved for both trichromatic and dichromatic observers. Color deficient observers benefit more than trichromats implying that the directional variability allows the recovery of the missing dimension in their colour vision systems. This study adds a new perspective to the classic trichromatic theory of human vision and emphasizes the importance of carefully considering the environments in which biological vision operates in daily life. It is striking that the physical directional variability available in natural lighting environments substantially mitigates the biological limitations of trichromacy or dichromacy.

When fixating an object, observers typically under or over-converge by a small amount, a phenomenon known as "fixation disparity". Fixation disparity is typically measured with physical fixation targets and dichotically presented nonius lines. Here we made fixation disparity measurements with an autostereoscopic display, varying the retinal eccentricity and disparity of the fixation targets. Measurements were made in a group of four practiced observers and in a group of thirteen experimentally naive observers. Fixation disparities with a zero-disparity target were in the direction of fixation behind the plane of the screen and the magnitude of the fixation disparity grew with the eccentricity of the fixation targets (1-5 deg in the practiced observers and 1 - 10 deg in the naive observers). Fixation disparity also increased with increasing disparity of the targets, especially when they were presented at crossed disparities. Fixation disparities were larger overall for naive observers who additionally did not converge in front of the screen when vergence demand was created by crossed disparity fusion locks presented at 5 and 10 deg eccentricities.

Judging poses, sizes and shapes of objects accurately is necessary for organisms and machines to operate successfully in the world. Retinal images of 3D objects are mapped by the rules of projective geometry, and preserve the invariants of that geometry. Since Plato, it has been debated whether geometry is innate to the human brain, and Poincare and Einstein thought it worth examining whether formal geometry arises from experience with the world. We examine if humans have learned to exploit projective geometry to estimate sizes and shapes of objects in 3D scenes.\n\nNumerous studies have examined size invariance as a function of physical distance, which changes scale on the retina, but surprisingly, possible constancy or inconstancy of relative size seems not to have been investigated for object pose, which changes retinal image size differently along different axes. We show systematic underestimation of length for extents pointing towards or away from the observer, both for static objects and dynamically rotating objects. Observers do correct for projected shortening according to the optimal back-transform, obtained by inverting the projection function, but the correction is inadequate by a multiplicative factor. The clue is provided by the greater underestimation for longer objects, and the observation that they appear more slanted towards the observer. Adding a multiplicative factor for perceived slant in the back-transform model provides good fits to the corrections used by observers. We quantify the slant illusion with relative slant measurements, and use a dynamic demonstration to show the power of the slant illusion.\n\nIn biological and mechanical objects, distortions of shape are manifold, and changes in aspect ratio and relative limb sizes are functionally important. Our model shows that observers try to retain invariance of these aspects of shape to 3D rotation by correcting retinal image distortions due to perspective projection, but the corrections can fall short. We discuss how these results imply that humans have internalized particular aspects of projective geometry through evolution or learning, and how assuming that images are preserving the continuity, collinearity, and convergence invariances of projective geometry, supplements the Generic Viewpoint assumption, and simply explains other illusions, such as Ames Chair.

Current assessments of interocular interactions in amblyopia use rivalrous stimuli, with conflicting stimuli in each eye, which does not reflect vision under typical circumstances. Here we measure interocular interactions in observers with amblyopia, strabismus with equal vision, and controls using a non-rivalrous stimulus. Observers used a joystick to continuously report perceived contrast of dichoptic grating stimuli, identical except that the stimulus was contrast-modulated independently in each eye over time. Consistent with previous studies, a model predicting the time-course of perceived contrast found increased amblyopic eye attenuation, and reduced contrast normalization of the fellow eye by the amblyopic eye, in amblyopic participants compared to controls. However, these suppressive interocular effects were weaker than those found in previous studies, suggesting that rivalrous stimuli may overestimate the effects of amblyopia on interocular interactions during naturalistic viewing conditions.

Fine spatial vision relies on the foveola, the 1-degree retinal region with highest cone density. Despite its importance, the relationship between retinal anatomy, fixational behavior, and visual perception in the foveola is not fully understood. Using an Adaptive Optics Scanning Light Ophthalmoscope for high-resolution retinal imaging and stimulation, we studied the effect of a simulated subfoveolar ({approx}0.03 degrees2) scotoma on fine spatial vision and fixation behavior in healthy observers. Our findings show that the visuomotor system adapts to the scotoma with striking precision by shifting the preferred locus of fixation in a systematic fashion by minute ({approx}5 arcmin) amounts to bring stimuli into a region of visibility. These results reveal an unprecedented level of fine-scale plasticity in the human visuomotor system. Interestingly, this new retinal locus of fixation is characterized by lower cone density among those surrounding the scotoma, indicating that factors beyond spatial sampling maximization influence these fine-scale adjustments.

Binocular fusion relies on matching points in the two eyes that correspond to the same physical feature in the world. However, not all world features are binocularly visible. In particular, at depth edges parts of a scene are often visible to only one eye (so-called half occlusions). Accurate detection of these monocularly visible regions is likely to be important for stable visual perception. If monocular regions are not detected as such, the visual system may attempt to binocularly fuse non-corresponding points, which can result in unstable percepts. We investigated the hypothesis that the visual system capitalizes upon statistical regularities associated with depth edges in natural scenes to aid binocular fusion and facilitate perceptual stability. By sampling from a large set of stereoscopic natural image patches, we found evidence that monocularly visible regions near depth edges in natural scenes tend to have features more visually similar to the adjacent binocularly visible background region than to the adjacent binocularly visible foreground. The generality of these results was supported by a parametric study of three-dimensional (3D) viewing geometry in simulated environments. In two perceptual experiments, we examined if this statistical regularity may be leveraged by the visual system. The results show that perception tended to be more stable when the visual properties of the depth edge were statistically more likely. Exploiting regularities in natural environments may allow the visual system to facilitate fusion and perceptual stability of natural scenes when both binocular and monocular regions are visible. PrecisWe report an analysis of natural scenes and two perceptual studies aimed at understanding how the visual statistics of depth edges impact perceptual stability. Our results suggest that the visual system exploits natural scene regularities to aid binocular fusion and facilitate perceptual stability.

Human viewers are able to perform tasks that depend on accurate estimates of surface reflectance, even across large changes in illumination and context. This is a remarkable ability, and successful image-computable models of how the visual system achieves this have remained elusive. Recently, deep convolutional neural networks (CNNs) have been developed that are adept at estimating surface reflectance. Here we evaluated one such network as a starting point for a new model of human lightness perception by testing whether it was susceptible to a range of classic lightness illusions. We implemented a CNN and trained it via supervised learning to estimate surface reflectance at each pixel in grayscale, rendered images of geometric objects. We examined the networks output on several illusions, including the argyle, Koffka, snake, simultaneous contrast, Whites, and checkerboard illusions, as well as control figures. We included variants where low-luminance regions important to the illusions were generated either by low reflectance or by cast shadows. For comparison, we carried out a lightness matching experiment with human observers using the same stimuli, and also examined the outputs of three classic lightness and brightness models. The CNN largely removed lighting effects such as shading and shadows, and produced good reflectance estimates on a test set. It also qualitatively predicted the illusions perceived by humans in most cases, the exceptions being Whites and checkerboard illusions. The CNN outperformed classical models, both at estimating reflectance and at tracking human lightness matches. These findings support a normative view of lightness perception and highlight the promise of deep learning models in this area.

Temporal differences in visual information processing between the eyes can cause dramatic misperceptions of motion and depth. Processing delays between the eyes cause the Pulfrich effect: oscillating targets in the frontal plane are misperceived as moving along near-elliptical motion trajectories in depth (Pulfrich, 1922). Here, we explain a previously reported but poorly understood variant: the anomalous Pulfrich effect. When this variant is perceived, the illusory motion trajectory appears oriented left- or right-side back in depth, rather than aligned with the true direction of motion. Our data indicate that this perceived misalignment is due to interocular differences in neural temporal integration periods, as opposed to interocular differences in delay. For oscillating motion, differences in the duration of temporal integration dampen the effective motion amplitude in one eye relative to the other. In a dynamic analog of the Geometric effect in stereo-surface-orientation perception (Ogle, 1950), the different motion amplitudes cause the perceived misorientation of the motion trajectories. Forced-choice psychophysical experiments, conducted with either different spatial frequencies and/or different onscreen motion damping in the two eyes, show that the perceived misorientation in depth is associated with the eye having greater motion damping. A target-tracking experiment provided more direct evidence that the anomalous Pulfrich effect is caused by interocular differences in temporal integration and delay. These findings highlight the computational hurdles posed to the visual system by temporal differences in sensory processing. Future work will explore how the visual system overcomes these challenges to achieve accurate perception.

Color constancy allows us to perceive stable object colors under different lighting conditions by reducing the impact of lighting. Information about illuminant color could be derived from a white surface or a specular highlight. The "brightest is white" heuristic has been frequently incorporated in illumination estimation models, to identify illuminant color. Here, we tested an alternative hypothesis: we use structured changes in the proximal image to identify highlight regions, even when they are not the brightest elements in the scene. In computer-rendered scenes, we varied the reliability of "brightest element" and "highlight geometry" cues, testing their effect on a color constancy task. Each scene had a single spherical surface lit by several point lights with identical spectral properties. The surface had a uniform spectral reflectance but a noise texture that attenuated the reflectance by a variable scale factor. We tested three levels of specularity: zero (matte), low, and mid. Observers watched a 1.5-second animation and responded if color changes were due to illuminant or material changes. Discrimination performance for matte surfaces was nearly at chance level, as predicted. However, as specularity increased, performance improved significantly. Observers outperformed an ideal observer model who relied solely on the brightest element. Notably, when the specular region appeared on a dark part of the texture, observer performance improved even more--even though the brightest element heuristic would predict a decrease. When specular geometries were difficult to identify due to phase scrambling, observer performance significantly dropped. These results suggest that we do not simply rely on the brightest element, but rather utilize regularities of diffuse and specular components of the proximal image to solve surface and illuminant ambiguities.

Luminance can vary widely when scanning across a scene, by up to 10^9 to 1, requiring multiple normalizing mechanisms spanning from the retina to cortex to support visual acuity and recognition. Vision models based on standard dynamic range luminance contrast ratios below 100 to 1 have limited ability to generalize to real-world scenes with contrast ratios over 10,000 to 1 (high dynamic range [HDR]). Understanding and modeling brain mechanisms of HDR luminance normalization is thus important for military applications, including automatic target recognition, display tone mapping, and camouflage. Yet, computer display of HDR stimuli was until recently unavailable or impractical for research. Here we describe procedures for setup, calibration, and precision check of an HDR display system with over 100,000 to 1 luminance dynamic range (650-0.0065 cd/m^2), pseudo 11-bit grayscale precision, and 3-ms temporal precision in the MATLAB/Psychtoolbox software environment. The setup is synchronized with electroencephalography and IR eye-tracking measurements. We report measures of HDR visual acuity and the discovery of a novel phenomenon--that abrupt darkening (from 400 to 4 cd/m^2) engages contextual facilitation, distorting the perceived orientation of a high-contrast central target. Surprisingly, the facilitation effect depended on luminance similarity, contradicting both classic divisive and subtractive models of contextual normalization.

The ability of humans to discriminate and identify spatial patterns varies across the visual field, and is generally worse in the periphery than in the fovea. This decline in performance is revealed in many kinds of tasks, from detection to recognition. A parsimonious hypothesis is that the representation of any visual feature is blurred (spatially averaged) by an amount that differs for each feature, but that in all cases increases with eccentricity. Here, we examine models for two such features: local luminance and spectral energy. Each model averages the corresponding feature in pooling windows whose diameters scale linearly with eccentricity. We performed perceptual experiments with synthetic stimuli to determine the largest window scaling for which human and model discrimination abilities match (the "critical" scaling). We used much larger stimuli than those of previous studies, subtending 53.6 by 42.2 degrees of visual angle. We found that the critical scaling for the luminance model was approximately one-fourth that of the energy model and, consistent with earlier studies, that the estimated critical scaling value was smaller when discriminating a synthesized stimulus from a natural image than when discriminating two synthesized stimuli. Moreover, we found that initializing the generation of the synthesized images with natural images reduced the critical scaling value when discriminating two synthesized stimuli, but not when discriminating a synthesized from a natural image stimulus. Together, the results show that critical scaling is strongly affected by the image statistic (pooled luminance vs. spectral energy), the comparison type (synthesized vs. synthesized or synthesized vs. natural), and the initialization image for synthesis (white noise vs natural image). We offer a coherent explanation for these results in terms of alignments and misalignments of the models with human perceptual representations.

Viewing repetitive striped patterns can induce pattern glare, experienced as visual discomfort (VD). While previous studies examined either pupillary responses or VD separately, few have investigated how they covary or evolve with repeated exposure. This study tested whether pupillary dynamics could serve as an objective "aversometer" -- a physiological marker of individual visual sensitivity beyond subjective reports. Across four experiments (preliminary: n = 97; main: n = 70 for spatial frequency, n = 46 for central field size, n = 36 for central blank, with partial overlap), we manipulated spatial frequency, central field size, and surround field size of square-wave gratings (0.5-3 s) while measuring both discomfort and pupil size. Higher spatial frequencies and larger pattern areas elicited stronger pupillary constriction and greater discomfort, whereas repeated exposures produced cumulative increases in discomfort and decreases in baseline pupil size, consistent with visual strain rather than adaptation. To assess the potential of pupillometry as an aversometer, we examined individual differences in the main spatial-frequency experiment (controlled viewing distance, n = 42). A paradoxical pattern emerged: within participants, stronger stimuli produced greater constriction, but individuals with higher overall discomfort showed weaker constriction and stronger late redilation. Similar dissociations between subjective sensitivity and pupillary responses have been noted in studies of light-induced discomfort, suggesting that related mechanisms may contribute, although their specific physiological basis remains unclear. Overall, our findings clarify how pattern-induced discomfort evolves over time and across individuals and highlight pupillometrys potential as a sensitive, objective tool for assessing visual sensitivity. HighlightsO_LIStriped patterns systematically increased discomfort and pupillary constriction C_LIO_LIRepeated exposure led to progressive discomfort and shrinking baseline pupil C_LIO_LIAmong high-sensitivity participants, weaker constriction and stronger redilation appeared C_LIO_LIThe paradox may reflect interindividual autonomic differences under visual stress C_LIO_LIPupillometry shows promise as an objective marker of visual sensitivity C_LI

Research suggests that spatial frequency (SF) channels in the visual system operate with a degree of independence. However, the independence model has been questioned by evidence of non-additive effects in compound gratings, indicating complex interactions between SF channels. These studies, however, typically employ artificial stimuli, leaving questions about SF processing in natural images. Efficient Coding hypothesis, which posits that the visual system minimizes redundancy and retains relevant information, predicts a dependence between HSF and LSF. In this study, we examined interactions between LSF and HSF using natural and phase-scrambled images to explore SF integration during perception. Participants completed an SF identification task, using both natural and scrambled images to isolate the role of phase alignment. Our results indicate that HSF and LSF interact primarily in phase-aligned conditions, with phase scrambling driving independent processing of two SFs and reducing error rates. These findings suggest that phase alignment enhances perceptual efficiency, facilitating a trade-off between accuracy and redundancy reduction in natural scene processing.

Notch filters can alter color contrasts by selectively filtering different spectral bands of the stimulus and have been developed to enhance reddish-greenish contrasts for color-deficient observers with anomalous trichromacy. We examined the effects of such filters on color salience for normal trichromats, using a visual search paradigm where the task was to locate a color target superimposed on a variegated chromatic background, similar to foraging for fruits among foliage. Background colors varied along a bluish-yellowish or purpliish to yellow-green (short-wave cone isolating) axis, roughly spanning the range of dominant color variations in arid or lush environments. Target colors sampled a wide range of hue angles and contrasts. Testing was conducted on a computer monitor, with the filter effects simulated by calculating corresponding chromaticities with or without the filter for naturalistic (Munsell) reflectances. The filter evaluated (Enchroma SuperX(R) glasses) was designed to increase color contrast along a magenta-green axis. Consistent with this, search times for targets on the blue-yellow background were significantly faster for the filter condition, because the filter increased the target-background color difference. Alternatively, overall differences in search times were not observed for the S-cone background. The differences on the two backgrounds could be qualitatively accounted for by the relative salience of the stimuli predicted by a perceptual color space (CIELAB). Our results demonstrate the efficacy of the filters for enhancing visual performance for normal trichromats and naturalistic tasks, and illustrate how these effects depend on the potential color characteristics of the environment.

Contrast modulation (CM) stimuli have been previously used to reveal nonlinear contributions of Y-like retinal ganglion cells (RGCs) such as parasol cells to cortical responses and perception. To test whether CMs are selectively processed within the magnocellular pathway, we assessed envelope motion discrimination and detection for achromatic (yellow-black) and chromatic (red-green) CMs in the presence of luminance masking noise to disrupt luminance-based mechanisms of motion processing. Compared to achromatic CMs, perception of chromatic CMs was more sensitive to luminance masking noise, suggesting that CM envelope motion perception relied predominantly on luminance signals. Specifically, envelope motion discrimination performance was better maintained for achromatic CMs than chromatic CMs, even at high masking noise levels. Notably, luminance masking noise greatly impaired envelope direction discrimination for chromatic CMs but had minimal impact on their detection, suggesting that chromatic aberrations may enhance envelope motion perception for chromatic CMs by introducing luminance signals. These findings collectively emphasize that CM stimuli selectively activate Y-like/parasol RGCs of the retino-geniculate magnocellular pathway, underscoring their potential to specifically target this pathway. This might have clinical advantages for early diagnosis in disorders such as glaucoma or dyslexia, where magnocellular pathway dysfunction is predominant.

The letter acuity impairment in the amblyopic eye often exceeds predictions made from the cut-off spatial frequency for grating detection. Spatial scrambling in the amblyopic eyes projections to the visual cortex has been proposed to bear some responsibility for this additional deficit. Using a novel stimulus algorithm that creates spatially scrambled bandpass letters, we generated stimuli simulating either: i) "cortical scrambling" at the output of oriented model "simple cells", or ii) "subcortical scrambling" of isotropic subunits that combine to form these simple cells. We also investigated a more conventional "noise masking" with bandpass noise. We performed two bandpass letter identification experiments, equating the stimuli shown to each eye by normalising either: i) their contrast, presenting them at four times their contrast detection threshold; or ii) their spatial scale, presenting them at twice the participants acuity threshold for each eye. At the group level, we found that the amblyopic eye is less efficient at performing letter identification in bandpass noise. We did not find an overall significant difference with either scrambling type when comparing efficiency between the amblyopic and fellow eye, but we did find such a difference when partitioning our participants by their stereopsis ability. In further analyses of the pattern of mistakes, we found the amblyopic eye shows a distinctive behaviour which correlates with the acuity deficit for both types of scrambling. These results demonstrate that our scrambled stimuli interrogate a component of amblyopic vision that is functionally distinct from that addressed by contrast noise masking.

We found a series of new illusions, in which actions performed near a random white-noise display lead to the perception that the display is altered interactively with the observers actions. The perceptions resemble interactions with a box of magnetic sand, where the hand can leave traces, or attract and repulse grains in its vicinity. 1) the observer puts a finger very close to a dynamic noise display, slowly moving as though drawing a letter or a shape. A trace appears left in the fingers path, decaying within 500 ms or so. 2) When the observer moves their palm toward and away from the display, opening and closing their fingers as if grabbing and releasing grains of sand, the random dots appear as though they were magnetically attracted to or repelled by the fingers. 3) When an open hand close to the display is slowly moved back and forth laterally, the nearby dots appear to get attracted to or captured by the fingers and thus appear to move with them. 4) The same kind of action capture occurs even when the hand is not visible, moving behind the display. These illusions are robust across a wide range of parameters, including frame rate, luminance contrast, dot size (spatial frequency), and finger movement factors. Inter-subject variability is not correlated across illusion types, and the illusions also diverge in behavior across dynamic and static noise conditions. This indicates that multiple mechanisms are involved to different extents across illusions. Several known visual motion detectors and other low-level mechanisms may be involved in seeding the perceptual phenomena. However, a complete explanation would require mechanisms of action capture, whereby the internal model of the persons actions and their predicted consequences organizes visual attention and processing of the random stimulus components.